Robot system

ABSTRACT

A robot system can grasp and take out one of a plurality of workpieces placed in a basket-like container by a hand mounted at the forward end of a robot arm. The workpiece is detected by a visual sensor, and the robot is controlled depending on a position and an orientation of the workpiece. When a problem such as interference or the like occurs, information relating to the problem is stored in a robot control unit or a visual sensor control unit. Information relating to the problem includes a predetermined amount of the latest data retrospectively traced from the time point of problem occurrence, a position which the robot has reached, the target position data, the content of the process executed by the visual sensor, and the detection result. When the problem is reproduced, these data are used to simulate the situation at the time of problem occurrence by using simulation unit. The situation at the time of problem occurrence can also be reproduced by using the actual robot without using the simulation unit.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a robot system employed for variousapplications which use an industrial robot and, more particularly, to atechnique for reproducing a situation in which an alarm is issued in thecase where the robot or an object mounted or held thereon clashes withsomething.

2. Description of the Related Art

If trouble occurs at a robot which performs an operation designated byan instruction such as a command in a taught operation program or thelike, it generally means that the external environment or the internalconditions which have been assumed for the operation of the robot havebeen changed. A robot system should be built such that such a situationcan rarely occur. However, in the event that such a situation occurs,the system is unavoidably stopped quickly to secure safety.

On the other hand, in an application, for example, for taking eachworkpiece out of bulk workpieces, the robot is required to perform theoperation according to the circumstances so that the robot changes itsmotion each time depending on irregular positions of the workpieces. Inother words, the external environment is so unstable in such anapplication that it is not uncommon to issue an alarm relating to thecontact or clash with comparative frequency. Therefore, if the system isstopped each time the alarm is issued, the working efficiency would begreatly deteriorated.

Techniques to avoid this inconvenience are disclosed, for example, inJapanese Patent No. 2662990 and Japanese Unexamined Patent PublicationNo. 2000-263481. The techniques described in these two patent documentsconcern a method of automatically restoring the system upon occurrenceof a trouble, and these techniques make it possible to reduce thefrequency at which the system is stopped and to improve the operatingefficiency. However, there is raised a problem that a desire to know asituation in which the alarm is issued cannot be easily satisfied.

On the other hand, the techniques for knowing the temporally pastphenomenon and the conditions at a spatially remote place are described,for example, in Japanese Unexamined Patent Publications No. 10-309683,No. 2001-179668 and No. 2001-277163.

The Japanese Unexamined Patent Publication No. 10-309683, which relatesto a robot and a method of analyzing an error caused by the robot,discloses the technique to record-data relating to a control for amotor, such as the operating waveform or the like data of the motor, forthe purpose of subsequent analysis.

Japanese Unexamined Patent Publication No. 2001-179668, which relates toa robot system for coping with accident, discloses a technique formonitoring a circumstance of an accident at an remote place using amonitoring device, such as a camera or the like, and a robotmanipulator. Specifically, the point of this technique is to prepare amonitoring device operable at a remote place to grasp the circumstanceof the remote place.

Further, Japanese Unexamined Patent Publication No. 2001-277163 relatesto a robot controller and a robot control method. A feature of thetechnique described in this document is that a system behavior plan andthe result of operation based on the plan are stored in a log.Specifically, in this technique, information on the operation is loggedto permit subsequent analysis of an operational status of a mobilerobot.

In the event of some failure in the robot system, it is desirable totake a preventive measure to avoid such a failure in the future. Thisrequires analysis of the phenomenon. However, such an analysis isdifficult in the case where the system has the ability to automaticallyrestore itself without stopping the operation. The ability toautomatically restore itself, therefore, is not always contributive to areduction in the frequency of the trouble. In other words, the techniquedescribed in Japanese Patent No. 2662990 or Japanese Unexamined PatentPublication No. 2000-263481 is simply to automatically restore thesystem and provides no means for subsequent analysis of the cause of thetrouble. Therefore, even though the system may be restoredautomatically, the system is finally required to be stopped in order toinvestigate the cause of the alarm, while no data useful to investigatethe cause of the alarm can be obtained.

Also, in the techniques described in Japanese Unexamined PatentPublications No. 10-309683 and No. 2001-277163, only specific data arecollected and it is difficult to reproduce the trouble that has actuallyoccurred. Further, the technique described in Japanese Unexamined PatentPublication No. 2001-179668 has no ability other than to confirm thephenomenon such as an accident at a remote place, or the like, in realtime.

SUMMARY OF THE INVENTION

Accordingly, an object of the present invention is to provide a robotsystem with a function to obtain information usable for reproducing atrouble which may occur (at the time of an alarm which may be issued)and a function of “reproducing the situation at the time of alarm” withan actual machine or on a simulator, thereby making it possible toreproduce the phenomenon with the actual machine when the system is notbusy or to virtually reproduce it at a remote place.

The present invention is intended to utilize these functions toeliminate the need to stop the system in operation, particularly foranalysis of the cause, thereby contributing to an improved operatingefficiency and system reliability.

The present invention can solve the above-mentioned problems byproviding a robot system with a function to record information on thesituation at the time of an alarm and to reproduce the situation at thetime of an alarm using the recorded information. The present inventioncan be implemented in several specific configurations.

A robot system according to a first aspect of the present inventionincludes a robot, and a visual sensor connected to the robot through adata transmitting means and for measuring one or both of a position andan orientation of an object in accordance with a command from the robotand transmitting a measurement result to the robot. The robot alsoincludes a means for sending, upon occurrence of a predefined alarm inthe robot in operation, a notification of the alarm occurrence, and anoperating condition of the robot at the time of alarm occurrence asrobot history information, to the visual sensor, a means for receivingthe robot history information from the visual sensor, and a means forreproducing the operating condition of the robot at the time of alarmoccurrence using the received robot history information. The visualsensor includes an accumulation means for accumulating measurementprocessing information obtained by measurement, a recording means forrecording, upon receipt of the notification of the alarm occurrence andthe robot history information from the robot, the robot historyinformation, while at the same time selecting measurement processinginformation associated with the robot operation at the time of alarmoccurrence from the measurement processing information accumulated inthe accumulation means and recording the selected measurement processinginformation as sensor history information, a means for sending the robothistory information recorded in the recording means to the robot, and ameans for reproducing a processing process of the visual sensor usingthe sensor history information.

A robot system according to a second aspect of the present inventionincludes a robot, and a visual sensor connected to the robot through adata transmitting means for measuring one or both of a position and anorientation of an object in accordance with a command from the robot andtransmitting a measurement result to the robot. The robot also includesa means for temporarily storing a predetermined amount of the latestexecuted operation commands as executed command information, a means forsending, upon occurrence of a predefined alarm in the robot inoperation, a notification of an alarm occurrence, the executed commandinformation, and an operating condition of the robot at the time ofalarm occurrence as robot history information, to the visual sensor, ameans for receiving the executed command information and the robothistory information from the visual sensor, and a means for reproducingthe operating condition of the robot before and at the time of alarmoccurrence using the received executed command information and robothistory information. The visual sensor also includes an accumulationmeans for accumulating measurement processing information obtained bymeasurement, a recording means for recording, upon receipt of thenotification of the alarm occurrence, the executed command informationand the robot history information from the robot, the executed commandinformation and the robot history information while, at the same time,selecting measurement processing information associated with the robotoperation at the time of alarm occurrence from the measurementprocessing information accumulated in the accumulation means andrecording the selected measurement processing information as sensorhistory information, a means for sending the executed commandinformation and robot history information recorded in the recordingmeans to the robot, and a means for reproducing a processing process ofthe visual sensor using the sensor history information.

A robot system according to a third aspect of the present inventionincludes a robot, and a simulation unit connected to the robot through acommunication line and for simulating operations of the robot. The robotalso includes a means for recording, upon occurrence of a predefinedalarm in the robot in operation, an operating condition of the robot atthe time of alarm occurrence as robot history information, and a meansfor transmitting the robot history information through the communicationline to the simulation unit. The simulation unit also includes a meansfor receiving the robot history information transmitted from the robot,and a means for reproducing the operating condition of the robot at thetime of alarm occurrence by simulation using the received robot historyinformation.

A robot system according to a fourth aspect of the present inventionincludes a robot, and a simulation unit connected to the robot through acommunication line and for simulating operations of the robot. The robotalso includes a means for temporarily storing a predetermined amount ofthe latest executed operation commands as executed command information,a means for recording, upon occurrence of a predefined alarm in therobot in operation, an operating condition of the robot at the time ofalarm occurrence as robot history information, and a means fortransmitting the executed command information and the robot historyinformation through the communication line to the simulation unit. Thesimulation unit also includes a means for receiving the executed commandinformation and robot history information transmitted from the robot,and a means for reproducing the operating condition of the robot beforeand at the time of alarm occurrence by simulation using the receivedexecuted command information and robot history information.

In all of the first to fourth embodiments described above, the robothistory information preferably includes information representing atleast one of a position which the robot has reached, a target positionat which the robot aimed, an interpolation mode used by the robot tooperate, an alarm generated in the robot, and the time of the alarmoccurrence.

A robot system according to a fifth aspect of the present inventionincludes a robot, a visual sensor connected to the robot through a datatransmitting means and for measuring one or both of a position and anorientation of an object in accordance with a command from the robot andtransmitting a measurement result to the robot, and a simulation unitconnected to the robot through a communication line and for simulatingone or both of operations of the robot and processes of the visualsensor. The robot also includes a means for notifying, upon occurrenceof a predefined alarm in the robot in operation, an alarm occurrence tothe visual sensor and recording an operating condition of the robot atthe time of alarm occurrence as robot history information, a means forreceiving measurement processing information of the visual sensorassociated with the robot operation at the time of alarm occurrence fromthe visual sensor and recording the measurement processing informationas sensor history information, and a means for transmitting the robothistory information and the sensor history information through thecommunication line to the simulation unit. The visual sensor alsoincludes an accumulation means for accumulating the measurementprocessing information obtained by measurement, and a means forselecting, upon receipt of the notification of the alarm occurrence fromthe robot, the measurement processing information associated with therobot operation at the time of alarm occurrence from the measurementprocessing information accumulated in the accumulation means, andtransmitting the selected measurement processing information to therobot. The simulation unit also includes a means for receiving the robothistory information and sensor history information transmitted from therobot, a means for reproducing the operating condition of the robot atthe time of alarm occurrence by simulation using the received robothistory information, and a means for reproducing the processing processof the visual sensor by simulation using the received sensor historyinformation.

A robot system according to a sixth aspect of the present inventionincludes a robot, a visual sensor connected to the robot through a datatransmitting means and for measuring one or both of a position and anorientation of an object in accordance with a command from the robot andtransmitting a measurement result to the robot, and a simulation unitconnected to the robot through a communication line and for simulatingone or both of operations of the robot and processes of the visualsensor. The robot also includes a means for temporarily storing apredetermined amount of the latest executed operation commands asexecuted command information, a means for notifying, upon occurrence ofa predefined alarm in the robot in operation, an alarm occurrence to thevisual sensor and recording an operating condition of the robot at thetime of alarm occurrence as robot history information, a means forreceiving measurement processing information of the visual sensorassociated with the robot operation at the time of alarm occurrence fromthe visual sensor and recording the measurement processing informationas sensor history information, and a means for transmitting the executedcommand information, the robot history information and the sensorhistory information through the communication line to the simulationunit. The visual sensor also includes an accumulation means foraccumulating the measurement processing information obtained bymeasurement, and a means for selecting, upon receipt of the notificationof the alarm occurrence from the robot, the measurement processinginformation associated with the robot operation at the time of alarmoccurrence from the measurement processing information accumulated inthe accumulation means and transmitting the selected measurementprocessing information to the robot. The simulation unit also includes ameans for receiving the executed command information, the robot historyinformation and the sensor history information transmitted from therobot, a means for reproducing the operating condition of the robotbefore and at the time of alarm occurrence by simulation using thereceived executed command information and robot history information, anda means for reproducing the processing process of the visual sensor bysimulation using the received sensor history information.

A robot system according to a seventh aspect of the present inventionincludes a robot, a visual sensor connected to the robot through a datatransmitting means and for measuring one or both of a position and anorientation of an object in accordance with a command from the robot andtransmitting a measurement result to the robot, and a simulation unitconnected to the robot sensor through a communication line and forsimulating one or both of operations of the robot and processes of thevisual sensor. The robot also includes a means for sending, uponoccurrence of a predefined alarm in the robot in operation, anotification of the alarm occurrence, and an operating condition of therobot at the time of alarm occurrence as robot history information, tothe visual sensor. The visual sensor also includes an accumulation meansfor accumulating measurement processing information obtained bymeasurement, a means for recording, upon receipt of the notification ofthe alarm occurrence and the robot history information from the robot,the robot history information, while at the same time selectingmeasurement processing information associated with the robot operationat the time of alarm occurrence from the measurement processinginformation accumulated in the accumulation means and recording theselected measurement processing information as sensor historyinformation, and a means for transmitting the robot history informationand the sensor history information through the communication line to thesimulation unit. The simulation unit also includes a means for receivingthe robot history information and sensor history information transmittedfrom the visual sensor, a means for reproducing the operating conditionof the robot at the time of alarm occurrence by simulation using thereceived robot history information, and a means for reproducing theprocessing process of the visual sensor by simulation using the receivedsensor history information.

A robot system according to an eighth aspect of the present inventionincludes a robot, a visual sensor connected to the robot through a datatransmitting means and for measuring one or both of a position and anorientation of an object in accordance with a command from the robot andtransmitting a measurement result to the robot, and a simulation unitconnected to the visual sensor through a communication line and forsimulating one or both of operations of the robot and processes of thevisual sensor. The robot also includes a means for temporarily storing apredetermined amount of the latest executed operation commands asexecuted command information, and a means for sending, upon occurrenceof a predefined alarm in the robot in operation, a notification of thealarm occurrence, the executed command information, and an operatingcondition of the robot at the time of alarm occurrence as robot historyinformation, to the visual sensor. The visual sensor also includes anaccumulation means for accumulating measurement processing informationobtained by measurement, a means for recording, upon receipt of thenotification of the alarm occurrence, the executed command informationand the robot history information from the robot, the executed commandinformation and the robot history information while, at the same time,selecting measurement processing information associated with the robotoperation at the time of alarm occurrence from the measurementprocessing information accumulated in the accumulation means andrecording the selected measurement processing information as sensorhistory information and, a means for transmitting the executed commandinformation, the robot history information and the sensor historyinformation through the communication line to the simulation unit. Thesimulation unit also includes a means for receiving the executed commandinformation, the robot history information and the sensor historyinformation transmitted from the visual sensor, a means for reproducingthe operating condition of the robot before and at the time of alarmoccurrence by simulation using the received executed command informationand robot history information, and a means for reproducing theprocessing process of the visual sensor by simulation using the receivedsensor history information.

In the first aspect or the second aspect or any one of the fifth toeighth aspects described above, the robot history information preferablyincludes information representing at least one of a position which therobot has reached, a target position at which the robot aimed, aninterpolation mode used by the robot to operate, an alarm generated inthe robot, and the time of alarm occurrence. Also, the sensor historyinformation preferably includes information representing at least one ofan image taken by the visual sensor, a content processed by the visualsensor, and a result of measurement processing in the visual sensor.

Also, in any of the third to eighth aspects described above, thecommunication line is preferably switchable between a connected stateand a disconnected state. Further, preferably, the simulation unit isadapted to be able to load a robot program describing the operation ofthe robot through the communication line, and has a function to change ateaching content of the robot program on the simulation unit and thenapply the changed robot program to the robot.

Preferably, the simulation unit is adapted to be able to load visualsensor processing parameters defining a process of the visual sensorthrough the communication line, and has a function to change a contentof the visual sensor processing parameters on the simulation unit andthen apply the changed visual sensor processing parameters to the visualsensor.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages will be describedin more detail below based on the preferred embodiments with referenceto the accompanying drawings, wherein:

FIG. 1 is a diagram illustrating an outline of a robot system accordingto a first embodiment of the present invention;

FIG. 2 is a diagram illustrating an outline of a robot system accordingto a second embodiment of the present invention;

FIG. 3 is a diagram illustrating an outline of a robot system accordingto a third embodiment of the present invention;

FIG. 4 is a diagram illustrating an outline of a robot system accordingto a fourth embodiment of the present invention;

FIG. 5 is a flowchart illustrating outlines of a process executed at thetime of alarm occurrence and a process executed in preparation for thealarm occurrence in the first embodiment;

FIG. 6 is a flowchart illustrating an outline of a process executed forreproducing the alarm in the first embodiment;

FIG. 7 is a flowchart illustrating outlines of a process executed at thetime of alarm occurrence and a process executed in preparation for thealarm occurrence in the second embodiment;

FIG. 8 is a flowchart illustrating an outline of a process executed forreproducing the alarm in the second embodiment;

FIG. 9 is a flowchart illustrating outlines of a process executed at thetime of alarm occurrence and a process executed in preparation for thealarm occurrence in the third embodiment;

FIG. 10 is a flowchart illustrating an outline of a process executed forreproducing the alarm in the third embodiment;

FIG. 11 is a flowchart illustrating outlines of a process executed atthe time of alarm occurrence and a process executed in preparation forthe alarm occurrence in the fourth embodiment; and

FIG. 12 is a flowchart illustrating an outline of a process executed forreproducing the alarm in the fourth embodiment.

DETAILED DESCRIPTION

FIGS. 1 to 4 show typical arrangements of robot systems according tofirst to fourth embodiments of the present invention, respectively. Thearrangement according to the first embodiment shown in FIG. 1corresponds to the first and second aspects described above; thearrangement according to the second embodiment shown in FIG. 2corresponds to the third and fourth aspects described above; thearrangement according to the third embodiment shown in FIG. 3corresponds to the fifth and sixth aspects described above; and thearrangement according to the fourth embodiment shown in FIG. 4corresponds to the seventh and eighth aspects described above.

Firstly, technical means commonly used in all or some of the embodimentswill be described below.

An exemplary application to which the robot system according to eachembodiment is applied is an operation in which each of a plurality ofworkpieces W placed in bulk in a basket-like container 10 is grasped andtaken out by a hand 13 mounted at the forward end of a robot arm 11. Inany of the embodiments, in order to control a robot (body mechanism) 15and the hand 13, a robot control unit 17 is arranged and connected tothe robot 15 and the hand 13 by means of a cable 19. The application inwhich one workpiece W is taken out of the bulk workpieces W is onlyillustrative and not intended to exclude other applications (welding byreal-time tracking using the visual sensor, for example) of the presentinvention.

In the first, third and fourth embodiments, in order to control therobot based on a position and an orientation of each workpiece W, avisual sensor (sensor head) 21 is mounted on the forward end of therobot arm 11 along with the hand 13. The visual sensor 21 is typically athree-dimensional visual sensor, such as a stereo type one having twovideo cameras or a combination of a video camera (photo detector) and aprojector for projecting a patterned light such as a slit light or aspot light.

In the embodiment using the visual sensor 21, a visual sensor controlunit 23 is provided for controlling an operation (including imaging,projection of patterned light, and detection of reflected light, etc.)of the sensor head 21 and processing signals (including video signals,and reflected light detection signals, etc.) obtained from the sensorhead 21 to conduct a three-dimensional measurement. The robot controlunit 17 and the visual sensor control unit 23 are connected to eachother through a data transmitting means 25 in order to exchange data andcommands between them.

In the second, third and fourth embodiments, a simulation unit 27 isprovided for reproducing a situation in which an alarm has occurred,without using the robot (the actual machine) 15. The simulation unit 27receives data required for simulation to reproduce the situation at thetime of alarm occurrence and transmits related commands. For thispurpose, the simulation unit 27 is connected to the robot control unit17 (in the second and third embodiments) or the visual sensor controlunit 23 (in the fourth embodiment) through a communication line 29.

The robot control unit 17 is a typically-used well-known type in which amain board is equipped with a CPU, a ROM, a RAM, a nonvolatile RAM, andothers. The system software for controlling the robot 15 is stored inthe ROM. This system software is typically copied into the RAM and thenexecuted by the CPU. Also, a robot program (operation program) includingoperation commands prepared by the user is stored in the nonvolatileRAM. The robot program is also typically copied into the RAM and thenexecuted by the CPU.

The main board of the robot control unit 17 is connected through a servoamplifier to a servo motor for driving the robot arm 11 on one hand anddirectly connected to an encoder of the servo motor and another signallines for I/O signals (external input/output signals) on the other hand.Also, a teaching panel (not shown) with a display is connected throughan input/output interface for the teaching panel to the main board ofthe robot control unit 17. A display may be disposed on the front of therobot control unit 17.

When an alarm occurs, information relating to the alarm is recorded inthe RAM or the nonvolatile RAM. As is well known, various alarms areused for various kind of troubles. A typical trouble against which analarm is issued includes interference (clash) between the robot arm 11or a robot equipment such as the visual sensor 21 or the hand 13, etc.and other objects.

In this exemplary application, a trouble (alarm) considered most liableto occur is interference between the wall of the basket-like container10 or the workpiece W to be held and another one of the surroundingworkpieces. A method of detecting an interference is well known andtherefore not described in detail below (for example, a method ofdetecting a clash by use of an external disturbance estimation parameterof a servo system can be used). Alarms against troubles other than theinterference includes an abnormal operation of the robot 15 such as alarge deviation from a taught path, a failure of the hand 13, and adetection failure of the visual sensor 21.

When these troubles occur, the following information is recorded in thenonvolatile RAM.

-   -   Alarm content: The type of the trouble is recorded as a        predetermined code or character. For example, codes are defined        as “001” for an interference, “010” for an abnormal operation of        the robot, “011” for a failure of the visual sensor, and “100”        for a failure of the hand.    -   Time of alarm occurrence    -   Position and orientation of the robot at time of alarm        occurrence    -   Destination (including a position and orientation upon arrival        at a target place) at which the robot aimed immediately before        alarm occurrence    -   Coordinate system used immediately before alarm occurrence: For        example, a code name of a user coordinate system or a tool        coordinate system, or a position and orientation data for an        origin of the coordinate system are recorded. These data include        a matrix representing a relation between a user coordinate        system and a reference world coordinate system, and a matrix        representing a relation between a tool coordinate system and a        mechanical interface coordinate system.    -   Interpolation mode used in the robot operation executed        immediately before alarm occurrence: axial interpolation, linear        interpolation, etc. are recorded as a predetermined code or        character. For example, the codes are defined as “001” for axial        interpolation and “010” for linear interpolation.    -   A predetermined amount of the latest executed commands of a        robot program: This is a data for allowing subsequent        reproduction of a path along which the robot 15 has moved for a        predetermined time period before the alarm occurrence. This data        is stored in the RAM or the nonvolatile RAM. The predetermined        amount is defined as, for example, “an information amount by        bits”, “the number of commands” or “the commands executed within        a predetermined time period in the past” and is preset in the        robot control unit 17.

The visual sensor control unit 23, as described above, controls anoperation (including imaging, projection of patterned light, anddetection of reflected light, etc.) of the sensor head 21, and processessignals (including video signals, and reflected light detection signals,etc.) obtained from the sensor head 21 to conduct a three-dimensionalmeasurement. A configuration, the basic functions, etc. of the visualsensor control unit 23 are well known and therefore not described indetail herein. The main board is equipped with a CPU, a ROM, a RAM, anonvolatile RAM, and a frame memory, etc., and a system software forcontrolling the visual sensor 21 is stored in the ROM.

This system software is typically copied into the RAM and then executedby the CPU. The main board is connected to the visual sensor (sensorhead) 21 through a visual sensor I/F, so that information of the imageprovided by the visual sensor 21, for example, is loaded into a framememory and processed by the CPU. The result of this processing is sentto the robot control unit 17 through the data transmitting means 25, andis used to control the operation of the robot 15.

Also, in the visual sensor control unit 23, the processes executed bythe visual sensor 21 and the used images are accumulated in the RAM orthe nonvolatile RAM (hard disk, etc.) as a log. When an alarm occurs inthe robot control unit 17, the visual sensor control unit 23 receives anotification of the alarm occurrence through the data transmitting means25 and collects sensor history information including informationassociated with the latest executed visual sensor processing and relatedinformation associated with the previously executed visual sensorprocessing. The collected sensor history information is recorded in theRAM or the nonvolatile RAM.

This sensor history information includes the following.

-   -   Content of the visual sensor processing: the content includes,        for example, history data representing a series of processes        from a detection of a workpiece W up to a measurement of a        position and an orientation of the workpiece W to be grasped.    -   Parameters used for the visual sensor processing: the parameters        include, for example, a electronic shutter speed of the imaging        means, a threshold value used for detecting a workpiece W, an        affine transformation parameter value for a model image used in        a matching method.    -   Time point at which the visual sensor processing is executed    -   Image taken by the visual sensor 21    -   Result of the visual sensor processing: the result includes, for        example, a position and an orientation of the workpiece W to be        held.

The visual sensor control unit 23 has attached thereto a monitor display(not shown), on which the various accumulated information describedabove, as well as the image being processed or immediately after beingprocessed, can be displayed. Also, with regard to the process describedlater, an item can be selected by the operator on the screen using apointer such as a mouse.

The simulation unit 27 used in the second to fourth embodiments containsgeometric data relating to the robot 15 and the peripheral devicesthereof and can simulate the operation of the robot 15. Although apersonal computer is a typical example, other conventional simulationunits may be used as the simulation unit 27. In the third and fourthembodiments, the simulation unit 27 can execute the same process as thevisual sensor control unit 23, and reproduce a specific imagingprocessing by receiving the image and the parameters necessary for imageprocessing through a communication line from the visual sensor controlunit 23.

The simulation unit 27 has attached thereto a monitor display (notshown), on which the contents of simulation and the various accumulatedinformation described above can be displayed. Also, with regard to theprocess described later, an item can be selected by the operator on thescreen using a pointer such as a mouse.

Based on the foregoing premise, an outline of a process in each of thefirst to fourth embodiments will be described with reference to theflowcharts shown in FIGS. 5 to 12. The example of the content of theinformation exchanged between the devices in the first to fourthembodiments are described above, and will not be described again below.Therefore, only the key points of each step will be described.

First Embodiment (FIG. 1)

In this embodiment, processes are executed in accordance with theflowchart shown in FIGS. 5 and 6. The flowchart of FIG. 5 represents anoutline of an event driven process executed in the visual sensor controlunit 23 and an interrupt process executed in the robot control unit 17at the time of alarm occurrence. The flowchart of FIG. 6, on the otherhand, represents an outline of a process executed in the robot controlunit 17 and the visual sensor control unit 23 to reproduce a situationat the time of alarm occurrence.

The points of each step executed in the robot control unit 17 are asfollows.

-   -   Step D1: Upon detection of an alarm, the robot control unit 17        determines an alarm ID and collects the robot history        information such as the alarm type. While the robot is        operating, a predetermined amount of the latest retrospective        data is always accumulated in the robot control unit 17 as robot        history information (the old data in the nonvolatile RAM is        erased and new data is added and stored from time to time).    -   Step D2: The alarm occurrence and the alarm ID are notified to        the visual sensor control unit 23.    -   Step D3: The robot history information collected in step D1 are        transmitted to the visual sensor control unit 23.    -   Step D4: The executed command information is transmitted to the        visual sensor control unit 23 to thereby complete the interrupt        process.

On the other hand, the points of each step executed in the visual sensorcontrol unit 23 are as follows.

-   -   Step E1: A start command from the robot control unit 17 to the        visual sensor 21 is awaited. After starting the robot 15, the        notification of the alarm occurrence can be received at the time        of alarm occurrence.    -   Step E2: If a received communication is a detection command, the        process proceeds to step E3, while if it is the notification of        the alarm occurrence, the process proceeds to step E8.    -   Step E3: A content of the process is selected depending on the        content of the detection command.    -   Step E4: An imaging trigger is sent to the visual sensor (sensor        head)21 to take an image.    -   Step E5: The image acquired by the visual sensor 21 is loaded        into the visual sensor control unit 23 and subject to an image        processing.    -   Step E6: The result of the image processing is transmitted to        the robot control unit 17.    -   Step E7: The image, the content of the process, the parameters,        the result of the processing, the time, etc. are recorded as        measurement processing information, and the process is returned        to step E1.    -   Step E8: The robot history information is received from the        robot control unit 17.    -   Step E9: The executed command information is received from the        robot control unit 17.    -   Step E10: The latest measurement processing information and the        related measurement processing information are selected and        stored as visual sensor history information.    -   Step E11: The robot history information and the executed command        information are grouped and recorded with the visual sensor        history information selected and stored in step E10. The alarm        ID determined in step D1 can be used for this purpose. When the        robot history information, the executed command information and        the visual sensor history information are recorded, the process        is completed.

The foregoing processes are an outline of the process executed inpreparation for alarm occurrence and an outline of the process executedat the time of alarm occurrence. The process for reproducing a situationat the time of alarm occurrence is executed in accordance with theflowchart of FIG. 6, using the data thus recorded.

The points of each step executed in the robot control unit 17 are asfollows.

-   -   Step F1: The robot history information is received from the        visual sensor control unit 23.    -   Step F2: The executed command information is received from the        visual sensor control unit 23.    -   Step F3: The robot history information and executed command        information received in steps F1 and F2 are used to reproduce        the operation from a robot position at a time point slightly        before an alarm occurrence to a robot position at the time of        alarm occurrence (the alarm occurrence point) (to reproduce the        situation at the time of alarm occurrence by using the actual        machine).

The points of each step executed by the visual sensor control unit 23are as follows.

-   -   Step G1: A list of the alarm history is displayed on the display        of the visual sensor control unit 23.    -   Step G2: The operator selects the alarm, which is desired to        reproduce, from the alarm history list on the screen. Thus, the        CPU of the visual sensor control unit 23 recognizes the alarm ID        of the selected alarm.    -   Step G3: The robot history information, the executed command        information and the visual sensor history information associated        with the selected alarm ID are selected (read from the memory).    -   Step G4: The robot history information is transmitted to the        robot control unit 17.    -   Step G5: The executed command information is transmitted to the        robot control unit 17.    -   Step G6: The visual sensor history information selected in step        G3 is used to reproduce the corresponding situation at the time        of alarm occurrence by way of image processing (display the        related image, the content of the process, etc. at the time of        alarm on the display of the visual sensor control unit 23), and        the process is completed.

Second Embodiment (FIG. 2)

In this embodiment, processes are executed in accordance with theflowcharts shown in FIGS. 7 and 8. The flowchart of FIG. 7 represents anoutline of an interrupt process executed in the robot control unit 17 atthe time of alarm occurrence. The flowchart of FIG. 8, on the otherhand, represents an outline of a process executed in the robot controlunit 17 and the simulation unit 27 to reproduce a situation at the timeof alarm occurrence.

The points of each step executed in the robot control unit 17 are asfollows.

-   -   Step H1: Upon detection of an alarm, the robot control unit 17        determines an alarm ID and collects the robot history        information such as the alarm type. While the robot is        operating, a predetermined amount of the latest retrospective        data is always accumulated in the robot control unit 17 as robot        history information (the old data in the nonvolatile RAM is        erased and new data is added and stored from time to time).    -   Step H2: The robot history information and the executed command        information are grouped and recorded. For this purpose, the        alarm ID determined in step H1 can be used. When the robot        history information and the executed command information are        recorded, the interrupt process is completed.

The foregoing processes are an outline of the process executed inpreparation for the alarm occurrence and an outline of the processexecuted at the time of alarm occurrence. The process for reproducing asituation at the time of alarm occurrence is executed in accordance withthe flowchart of FIG. 8, using the data thus recorded.

The points of each step executed in the robot control unit 17 are asfollows.

-   -   Step J1: A list of the alarm history is transmitted to the        simulation unit 27.    -   Step J2: The receipt of the selected alarm ID is awaited. When        it is received, the process proceeds to step J3.    -   Step J3: The robot history information and the executed command        information associated with the selected alarm ID are selected        (read from the memory).    -   Step J4: The robot history information and executed command        information selected in step J3 are transmitted to the        simulation unit 27, and the process is completed.

On the other hand, the points of each step executed in the simulationunit 27 are as follows.

-   -   Step K1: A list of the alarm history is received from the robot        control unit 17.    -   Step K2: The operator selects the alarm, which is desired to        reproduce, from the alarm history list on the screen. Thus, the        CPU of the simulation unit 27 recognizes the alarm ID of the        selected alarm.    -   Step K3: The selected ID alarm is transmitted to the robot        control unit 17.    -   Step K4: The robot history information and the executed command        information are received from the robot control unit 17.    -   Step K5: The robot history information and executed command        information received in step K4 are used to execute a simulation        for reproducing the operation from a robot position at a time        point slightly before the alarm occurrence to a robot position        at the time of alarm occurrence (alarm occurrence point) (for        reproducing the situation at the time of alarm occurrence by        using the simulation unit 27).

Third Embodiment (FIG. 3)

In this embodiment, processes are executed in accordance with theflowcharts shown in FIGS. 9 and 10. The flowchart of FIG. 9 representsan outline of an event driven process executed in the visual sensorcontrol unit 23 and an interrupt process executed in the robot controlunit 17 at the time of alarm occurrence. The flowchart of FIG. 10, onthe other hand, represents an outline of a process executed in the robotcontrol unit 17 and the simulation unit 27 to reproduce a situation atthe time of alarm occurrence.

The points of each step executed by the robot control unit 17 are asfollows.

-   -   Step L1: Upon detection of an alarm, the robot control unit 17        determines an alarm ID and collects the robot history        information such as the alarm type. While the robot is        operating, a predetermined amount of the latest retrospective        data is always accumulated in the robot control unit 17 as robot        history information (the old data in the nonvolatile RAM is        erased and new data is added and stored from time to time).    -   Step L2: The alarm occurrence and the alarm ID are notified to        the visual sensor control unit 23.    -   Step L3: The visual sensor history information is received from        the visual sensor control unit 23.    -   Step L4: The robot history information and the executed command        information are grouped and recorded with the visual sensor        history information received in step L3. For this purpose, the        alarm ID determined in step L1 can be used. When the robot        history information, the executed command information and the        visual sensor history information are recorded, the process is        completed.

On the other hand, the points of each step executed in the visual sensorcontrol unit 23 are as follows.

-   -   Step M1: A start command from the robot control unit 17 to the        visual sensor 21 is awaited. After starting the robot 15, the        notification of the alarm occurrence can be received at the time        of alarm occurrence.    -   Step M2: If a received communication is a detection command, the        process proceeds to step M3, while if it is the notification of        the alarm occurrence, the process proceeds to step M8.    -   Step M3: A content of the process is selected depending on the        content of the detection command.    -   Step M4: An imaging trigger is sent to the visual sensor (sensor        head) 21 to take an image.    -   Step M5: The image acquired by the visual sensor 21 is loaded        into the visual sensor control unit 23, and subject to an image        processing.    -   Step M6: The result of the image processing is transmitted to        the robot control unit 17.    -   Step M7: The image, the content of the process, the parameters,        the result of the processing, the time, etc. are recorded as        measurement processing information, and the process returns to        step M1.    -   Step M8: The latest measurement processing information and the        related measurement processing information are selected and        stored as visual sensor history information.    -   Step M9: The visual sensor history information is transmitted to        the robot control unit 17, and the process is completed.

The foregoing processes are an outline of the process executed inpreparation for alarm occurrence and an outline of the process executedat the time of alarm occurrence. The process for reproducing a situationat the time of alarm occurrence is executed in accordance with theflowchart of FIG. 10, using the data thus recorded.

The points of each step executed in the robot control unit 17 are asfollows.

-   -   Step P1: A list of the alarm history is transmitted to the        simulation unit 27.    -   Step P2: The receipt of the selected alarm ID is awaited. When        it is received, the process proceeds to step P3.    -   Step P3: The robot history information, executed command        information and visual sensor history information associated        with the selected alarm ID are selected (read from the memory).    -   Step P4: The robot history information, the executed command        information and the visual sensor history information are        transmitted to the simulation unit 27, and the process is        completed.

On the other hand, the points of each step executed in the simulationunit 27 are as follows.

-   -   Step Q1: A list of the alarm history is received from the robot        control unit 17.    -   Step Q2: The operator selects the alarm, which is desired to        reproduce, from the alarm history list on the screen. Thus, the        CPU of the simulation unit 27 recognizes the alarm ID of the        selected alarm.    -   Step Q3: The selected alarm ID is transmitted to the robot        control unit 17.    -   Step Q4: The robot history information, the executed command        information and the visual sensor history information are        received from the robot control unit 17.    -   Step Q5: The visual sensor history information received in step        K4 is used to reproduce the corresponding situation at the time        of alarm occurrence by way of the image processing (display the        related image, the content of the process, etc. at the time of        alarm occurrence on the display of the simulation unit 27).    -   Step Q6: The information received in step K4 is used to execute        a simulation for reproducing the operation from a robot position        at the time point slightly before alarm occurrence to a robot        position at the time of alarm occurrence (alarm occurrence        point) (for reproducing the situation at the time of alarm        occurrence by using the simulation unit 27).

Fourth Embodiment (FIG. 4)

In this embodiment, processes are executed in accordance with theflowcharts shown in FIGS. 11 and 12. The flowchart of FIG. 11 representsan outline of an event driven process executed in the visual sensorcontrol unit 23 and an interrupt process executed at the time of alarmoccurrence in the robot control unit 17. The flow chart of FIG. 12, onthe other hand, represents an outline of a process executed in thevisual sensor control unit 23 and the simulation unit 27 to reproduce asituation at the time of alarm occurrence.

The points of each step executed in the visual sensor control unit 23are as follows.

-   -   Step S1: A start command from the robot control unit 17 to the        visual sensor 21 is awaited. After starting the robot 15, a        notification of the alarm occurrence can be received at the time        of alarm occurrence.    -   Step S2: If a received communication is a detection command, the        process proceeds to step M3, while if it is the notification of        the alarm occurrence, the process proceeds to step S8.    -   Step S3: A content of the process is selected depending on the        content of the detection command.    -   Step S4: An imaging trigger is sent to the visual sensor (sensor        head) 21 to take an image.    -   Step S5: The image acquired by the visual sensor 21 is loaded        into the visual sensor control unit 23, and subject to an image        processing.    -   Step S6: The result of the image processing is transmitted to        the robot control unit 17.    -   Step S7: The image, the content of the process, the parameters,        the result of the processing, the time, etc. are recorded as        measurement processing information, and the process returns to        step S1.    -   Step S8: The robot history information is received from the        robot control unit 17.    -   Step S9: The executed command information is received from the        robot control unit 17.    -   Step S10: The latest measurement processing information and the        related measurement processing information are selected and        stored as visual sensor history information.    -   Step S11: The robot history information and the executed command        information are grouped and recorded with the visual sensor        history information selected and stored in step S10. For this        purpose, the alarm ID determined in the robot control unit 17        (step R1, described later) can be used. When the robot history        information, the executed command information and the visual        sensor history information are recorded, the process is        completed.

On the other hand, the points of each step executed in the robot controlunit 17 are as follows.

-   -   Step R1: Upon detection of an alarm, the robot control unit 17        determines an alarm ID and collects the robot history        information such as the alarm type. While the robot is        operating, a predetermined amount of the latest retrospective        data is always accumulated in the robot control unit 17 as robot        history information (the old data in the nonvolatile RAM is        erased and new data is added and stored from time to time).    -   Step R2: The alarm occurrence and the alarm ID are notified to        the visual sensor control unit 27.    -   Step R3: The executed command information is transmitted to the        visual sensor control unit 23, and the process is completed.

The foregoing processes are an outline of the process executed inpreparation for alarm occurrence and an outline of the process executedat the time of alarm occurrence. The process for reproducing a situationat the time of alarm occurrence is executed in accordance with theflowchart of FIG. 12, using the data thus recorded.

The points of each step executed in the simulation unit 27 are asfollows.

-   -   Step T1: A list of alarm history is received from the visual        sensor control unit 23.    -   Step T2: The operator selects the alarm, which is desired to        reproduce, from the alarm history list on the screen. Thus, the        CPU of the simulation unit 27 recognizes the alarm ID of the        selected alarm.    -   Step T3: The selected alarm ID is transmitted to the visual        sensor control unit 23.    -   Step T4: The robot history information, the executed command        information and the visual sensor history information are        received from the visual sensor control unit 23.    -   Step T5: The visual sensor history information received in step        T4 is used to reproduce the corresponding situation at the time        of alarm occurrence by way of image processing (display the        related image, the content of the process, etc. at the time of        alarm occurrence on the display of the simulation unit 27).    -   Step T6: The information received in step T4 is used to execute        a simulation for reproducing the operation from a robot position        at the time point slightly before the alarm occurrence to a        robot position at the time of alarm occurrence (alarm occurrence        point) (for reproducing the situation at the time of alarm        occurrence is reproduced by using the simulation unit 27).

On the other hand, the points of each step executed in the visual sensorcontrol unit 23 are as follows.

-   -   Step U1: A list of alarm history is transmitted to the        simulation unit 27.    -   Step U2: The receipt of the selected alarm ID is awaited. When        it is received, the process proceeds to step U3.    -   Step U3: The robot history information, executed command        information and visual sensor history information associated        with the selected alarm ID are selected (read from the memory).    -   Step U4: The robot history information, the executed command        information and the visual sensor history information are        transmitted to the simulation unit 27, and the process is        completed.

As described above, when an alarm relating to a clash or the likeoccurs, the robot system according to the present invention (such as arobot system for carrying out the operation of taking out a part frombulk parts, for example) stores an alarm information together with aposition and an orientation at the time of alarm occurrence, the targetposition and orientation at the time of alarm occurrence, the result ofthe image processing and the related data which has affected the robotoperation, and reproduces the alarm situation based on this information.The reproduction can be implemented by either the on-line reproductionby means of the actual robot machine or the off-line reproduction bymeans of the simulation unit (arranged at a remote place, for example).Also, this reproduction makes it possible not only to confirm thespecifics of the trouble, but also to know the detail of the situationof the robot (the orientation thereof at the time of alarm occurrenceand the orientation thereof in the process before alarm occurrence) atthe time of trouble occurrence, thereby facilitating the tracing of thecause and the removal of the problem. Further, in the case where thesimulation unit is used for reproducing the trouble situation bysimulation, the change of the robot program or the values of theparameters for image processing can be carried out on line to remove theproblem. Therefore, the preventive measure can be taken veryefficiently.

Although the present invention has been described based on theembodiments shown in the accompanying drawings, these embodiments areonly illustrative and not limitative. Accordingly, the scope of thepresent invention is limited by the appended claims, and the embodimentsof the present invention may be modified or changed without departingfrom the scope of the claims.

1. A robot system capable of reproducing a situation at the time ofalarm occurrence, said robot system comprising: a robot; and asimulation unit connected to said robot through a communication line andfor simulating operations of said robot, said robot comprising: a meansfor recording, upon occurrence of a predefined alarm in said robot inoperation, an operating condition of the robot at the time of alarmoccurrence as robot history information; and a means for transmittingthe robot history information through said communication line to saidsimulation unit, and said simulation unit comprising: a means forreceiving the robot history information transmitted from said robot; anda means for reproducing the operating condition of said robot at thetime of alarm occurrence by simulation using the received robot historyinformation.
 2. The robot system according to claim 1, wherein the robothistory information includes information representing at least one of aposition which said robot has reached, a target position at which saidrobot aimed, an interpolation mode used by said robot to operate, analarm generated in said robot, and a time of alarm occurrence.
 3. Therobot system according to claim 1, wherein said communication line isadapted to be able to switch between a connected state and adisconnected state.
 4. The robot system according to claim 1, whereinsaid simulation unit is adapted to be able to load a robot programdescribing the operation of said robot through said communication line,and has a function to change a teaching content of the robot program onsaid simulation unit and then apply the changed robot program to saidrobot.
 5. A robot system capable of reproducing a situation at the timeof alarm occurrence, said robot system comprising: a robot; and asimulation unit connected to the robot through a communication line andfor simulating operations of the robot, said robot comprising: a meansfor temporarily storing a predetermined amount of the latest executedoperation commands as executed command information; a means forrecording, upon the occurrence of a predefined alarm in said robot inoperation, an operating condition of said robot at the time of alarmoccurrence as robot history information; and a means for transmittingthe executed command information and the robot history informationthrough said communication line to said simulation unit, and saidsimulation unit comprising: a means for receiving the executed commandinformation and robot history information transmitted from said robot;and a means for reproducing the operating condition of said robotbefore, and at the time of, the alarm occurrence by simulation using thereceived executed command information and robot history information. 6.The robot system according to claim 5, wherein the robot historyinformation includes information representing at least one of a positionwhich said robot has reached, a target position at which said robotaimed, an interpolation mode used by said robot to operate, an alarmgenerated in said robot, and a time of alarm occurrence.
 7. The robotsystem according to claim 5, wherein said communication line is adaptedto be able to switch between a connected state and a disconnected state.8. The robot system according to claim 5, wherein said simulation unitis adapted to be able to load a robot program describing the operationof said robot through said communication line, and has a function tochange a teaching content of the robot program on said simulation unitand then apply the changed robot program to said robot.
 9. A robotsystem capable of reproducing a situation at the time of alarmoccurrence, said robot system comprising: a robot; a visual sensorconnected to said robot through a data transmitting means and formeasuring one or both of a position and an orientation of an object inaccordance with a command from said robot and transmitting a measurementresult to said robot; and a simulation unit connected to said robotthrough a communication line and for simulating one or both ofoperations of said robot and processes of said visual sensor, said robotcomprising: a means for notifying, upon occurrence of a predefined alarmin said robot in operation, an alarm occurrence to said visual sensorand recording an operating condition of said robot at the time of alarmoccurrence as robot history information; a means for receivingmeasurement processing information of said visual sensor associated withthe robot operation at the time of alarm occurrence from said visualsensor and recording the measurement processing information as sensorhistory information; and a means for transmitting the robot historyinformation and the sensor history information through saidcommunication line to said simulation unit, and said visual sensorcomprising: an accumulation means for accumulating the measurementprocessing information obtained by measurement; and a means forselecting, upon receipt of the notification of the alarm occurrence fromsaid robot, the measurement processing information associated with therobot operation at the time of alarm occurrence from the measurementprocessing information accumulated in said accumulation means, andtransmitting the selected measurement processing information to saidrobot, and said simulation unit comprising: a means for receiving therobot history information and sensor history information transmittedfrom said robot; a means for reproducing the operating condition of saidrobot at the time of alarm occurrence by simulation using the receivedrobot history information; and a means for reproducing the processingprocess of said visual sensor by simulation using the received sensorhistory information.
 10. The robot system according to claim 9, whereinthe robot history information includes information representing at leastone of a position which said robot has reached, a target position atwhich said robot aimed, an interpolation mode used by said robot tooperate, an alarm generated in said robot, and a time of alarmoccurrence, and wherein the sensor history information includesinformation representing at least one of an image taken by said visualsensor, a content processed by said visual sensor, and a result ofmeasurement processing in said visual sensor.
 11. The robot systemaccording to claim 9, wherein said communication line is adapted to beable to switch between a connected state and a disconnected state. 12.The robot system according to claim 9, wherein said simulation unit isadapted to be able to load a robot program describing the operation ofsaid robot through said communication line, and has a function to changea teaching content of the robot program on said simulation unit and thenapply the changed robot program to said robot.
 13. The robot systemaccording to claim 9, wherein said simulation unit is adapted to be ableto load visual sensor processing parameters defining a process of saidvisual sensor through said communication line, and has a function tochange a content of said visual sensor processing parameters on saidsimulation unit and then apply the changed visual sensor processingparameters to said visual sensor.
 14. A robot system capable ofreproducing a situation at the time of alarm occurrence, said robotsystem comprising: a robot; a visual sensor connected to said robotthrough a data transmitting means and for measuring one or both of aposition and an orientation of an object in accordance with a commandfrom said robot and transmitting a measurement result to said robot; anda simulation unit connected to said robot through a communication lineand for simulating one or both of operations of said robot and processesof said visual sensor, said robot comprising: a means for temporarilystoring a predetermined amount of the latest executed operation commandsas executed command information; a means for notifying, upon occurrenceof a predefined alarm in said robot in operation, an alarm occurrence tosaid visual sensor and recording an operating condition of said robot,at the time of alarm occurrence, as robot history information; a meansfor receiving measurement processing information of said visual sensorassociated with the robot operation at the time of alarm occurrence andrecording the measurement processing information as sensor historyinformation; and a means for transmitting the executed commandinformation, the robot history information and the sensor historyinformation through said communication line to said simulation unit,said visual sensor comprising: an accumulation means for accumulatingthe measurement processing information obtained by measurement; and ameans for selecting, upon receipt of the notification of the alarmoccurrence from said robot, the measurement processing informationassociated with the robot operation at the time of alarm occurrence fromthe measurement processing information accumulated in said accumulationmeans and transmitting the selected measurement processing informationto said-robot, and said simulation unit comprising: a means forreceiving the executed command information, the robot historyinformation and the sensor history information transmitted from saidrobot; a means for reproducing the operating condition of said robotbefore and at the time of alarm occurrence by simulation using thereceived executed command information and robot history information; anda means for reproducing the processing process of said visual sensor bysimulation using the received sensor history information.
 15. The robotsystem according to claim 14, wherein the robot history informationincludes information representing at least one of a position which saidrobot has reached, a target position at which said robot aimed, aninterpolation mode used by said robot to operate, an alarm generated insaid robot, and a time of alarm occurrence, and wherein the sensorhistory information includes information representing at least one of animage taken by said visual sensor, a content processed by said visualsensor, and a result of measurement processing in said visual sensor.16. The robot system according to claim 14, wherein said communicationline is adapted to be able to switch between a connected state and adisconnected state.
 17. The robot system according to claim 14, whereinsaid simulation unit is adapted to be able to load a robot programdescribing the operation of said robot through said communication line,and has a function to change a teaching content of the robot program onsaid simulation unit and then apply the changed robot program to saidrobot.
 18. The robot system according to claim 14, wherein saidsimulation unit is adapted to be able to load visual sensor processingparameters defining a process of said visual sensor through saidcommunication line, and has a function to change a content of saidvisual sensor processing parameters on said simulation unit and thenapply the changed visual sensor processing parameters to said visualsensor.